Talk:The Dark Priest and the Pagan Altar/@comment-30423180-20170519140538/@comment-28059753-20170527092543
Indeed, we've discussed this at length before; I just felt like I hadn't stated my case for quite a while, so I would restate it somewhere more visible. Sure I'm picking a convenient drop rate -- the most likely drop rate given my model. This is admittedly a slightly sticky point, as, even if my model is correct, I'm going to sometimes choose drop rates that fit the model better than reality (ie overfitting). I see this more as a lack-of-data issue, though, as given more data, the range of value which another model has to "fight" against becomes smaller. I could as easily accuse you of choosing a drop rate that is inconvenient for my model. I don't feel like the model should be discarded because one anonymous user with one post put some random number in my mouth whilst using poor spelling and grammar (no offense @anon). I'm not sure how to avoid this problem, but forcing an inconvenient drop rate seems extremely silly. There may be a lot of unknowns when dealing with unsolicited reports like Kutko's which is why I'm in favor of collecting solicited reports in a context where the results should avoid such issues. I don't think there is bias in my discounting such results sight-unseen. If there are problems with the model, they should appear in the solicited reports as well. It may take longer for them to appear, but there is no difficulty in trying to account for the possible biases of unsolicited reports. I may be handicapping myself by ignoring some data, but I think it is worth it so that I don't accidentally introduce bias due to handling such complicated factors incorrectly. I guess 1.8% (which I think is a low estimate because I was assuming 20%) can be viewed in light of the number of reports. Reports won't have exactly 1.8% of being less likely than 1.8% due to the discrete nature of the distribution (it will be less than 1.8% in general), but this problem should be reduced as the number of runs grows. Ignoring this problem, and considering reports of, say, >= 15 runs, the chance of at least one 1.8% result is 1 - (1 - 0.018)^5 = 8.7%. Unlikely, but not too surprising. Even taking 1% to help account for the discretization problem, it's still only 4.9%. That's maybe worrying, but not compelling, and not considering that's only for the very generous (IMO) 20% number. So, 1.8% compared to the number of runs being considered, and some target likelihood level, if you like. Anyway, I wasn't setting out to disprove your theory. I was just trying to examine it using what I consider to be fairly sound data that I thought was relevant to the question at hand. The type of analysis I was trying to do should show evidence of your theory if it is correct, no? At least with some probability. Eventually someone will report a serious outlier like Kutko apparently had. Or not. If it happens too many times compared to what the fixed drop rate model can account for, that may be good evidence that fixed drop rates are wrong. I have also done other analysis trying to examine whether a fixed drop rate is a good model (using 0x vs 1x vs 2x distribution of rubies in Phalanx 2 since it has quite a lot of data). I didn't see anything unusual here either. (But maybe your theory doesn't apply to challenges?) So, yes, I agree more data would be nice, but I'm trying to do impartial experiments (though I am admittedly skeptical) that should give some insight into fixed drop-rate vs non-fixed drop-rate.